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Purpose  and  Agenda 


Purpose:  To  brief  the  methods  of  a  structured  observational 
study  within  the  Future  Combat  Systems  (FCS)  Spin  Out  Early 
(SO-E)  Force  Development  Test  and  Evaluation  (FDTE)  in 
2009  and  cognitive  metrics  implications  for  mission  command 
(MC). 


Agenda 

•  Background. 

•  Methods. 

•  Implications  for  MC  Analysis. 


17  January  2012 
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The  Cognition  Problem  for  Mission  Command 


The  Army  plans  to  push  huge  amounts  of  information  across 
the  network  without  examining  the  cognitive  limitations  of 
Soldiers  to  make  effective  use  of  that  information.  The  Army 
is: 

•  Building  complex  technology  solutions. 

•  Pushing  MC  responsibilities  down  to  lower  echelons. 

•  Assuming  more  information  is  better. 


How  can  the  Army  measure  cognitive  limitations  and  feed  results  back 
into  DOTMLPF  solutions?  What  DOTMLPF  changes  do  we  need  to  do 

with  respect  to  workload  and  MCF? 


DOTMLPF  -  doctrine,  organization,  training,  materiel,  leadership  and  education,  personnel,  and  facilities 
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Objective  of  Cognitive  Research 


To  optimize  cognitive  task  performance  such  that  Soldiers,  Leaders, 
and  Teams  may  achieve  mission  success  operating  in  the  enhanced 
automated  MC  environment  of  the  future  force. 

Goals  for  Cognitive  Measurement: 

•  Creativity. 

-  Promotes  active  and  spontaneous  behavior  in  response  to  task 
demands. 

•  Real-time. 

-  Enables  synchronization  with  mission  events  and  the  flow  of 
information. 

•  Objective. 

-  Helps  shield  the  metric  from  bias. 

•  Unobtrusive. 

-  Preserves  the  integrity  of  the  event  and  the  Soldiers’  behavior. 

•  Diagnostic. 

-  Informs  designers  about  the  source  of  performance  issues. 
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Analysis  in  Relation  to  DOTMLPF 


Social  Domain 

Cultural  Awareness 


The  analytical  campaign  was 
structured  to  provide 
relevant  cognitive  analysis  to 
inform  DOTMLPF  solutions. 

CCRP  -  Command  and  Control  Research  Program 

DOD  -  Department  of  Defense 
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Organization 


Materiel 


Source:  TRAC-WSMR  Forward 


TRAC-WSMR  Forward  -  TRADOC  Analysis  Center-White  Sands  Missile  Range  (Forward) 
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•  Background. 

•  Methods. 
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Methods 


•  Observers  were  positioned  at  the  battalion  (BN)  tactical 
operations  center  (TOC)  and  company  (CO)  command  post 
on  Phase  2  Day  1. 

•  Information  about  workload  and  macrocognitive  functions 
(MCF)  was  collected  using  pen-and-paper  instrumentation. 

•  Observers  were  left  free  to  choose  whether  to  record  by 

major  evenl  or  by  regular  time  intervals. 

•  Subject  matter  experts  (SME)  observed: 

-One  CO  leader  at  the  command  post. 

-Collective  behaviors  at  the  BN  TOC. 

•  Intentional  variation  allowed  the  instrumentation  and  protocol 
to  be  piloted  in  a  number  of  ways. 


17  January  2012 


UNCLASSIFIED 


Potential  Discriminating  Metrics  of  Cognitive  Task  Performance  in  MC 


Assumptions  About  Workload  Measurement 


•  Every  task  that  a  Soldier  performs  will  demand  attention  and 
contribute  to  their  cognitive  workload. 

•  If  workload  is  either  too  high  or  too  low,  performance  suffers. 

•  Performance  can  be  predicted  from  a  diagnostic  measure  of 
workload. 

•  Workload  can  be  measured  through  repeatable,  behaviorally 
anchored  methods,  promoting  objectivity. 
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Workload  Scale* 


Code 

Label 

Description 

1 

Little  to  No 
Activity 

Level  of  activity  is  such  that  the  Soldier/operator  is  not  being 
engaged  at  a  high  enough  level  to  maintain  attention,  and  the 
Soldier/operator  may  drift  off  task,  or  do  other  tasks. 

Behavioral  indicators:  Station  not  manned,  operator  asleep, 
delays  in  response  to  messages,  workstation  may  be  manned 
by  others  while  Soldier/operator  is  on  long  breaks. 

2 

Low  Activity 

The  level  of  activity  does  not  fully  engage  the  Soldier/operator. 
Behavioral  indicators.  Tasks  are  completed  on  time. 

3 

Optimal* 

Activity 

All  tasks  are  being  achieved.  The  Soldier/operator  is  able  to 
schedule  tasks  so  that  all  task  are  achieved  in  a  timely  fashion, 
however  lower  priority  and  nonmission  critical  task  might  have 
some  delay. 

4 

High  Activity 

All  mission  critical  tasks  are  being  accomplished,  however 
some  nonmission  critical  tasks  are  being  dropped,  or  delayed 
to  a  later  time. 

5 

Overloaded 

Some  mission  critical  tasks  are  being  dropped,  or  delayed  so 
as  not  to  be  timely. 

System  development  should  target  the  middle  of  the  workload  scale. 


*Embrey,  D.,  Blackett,  C.,  Marsden,  P.,  &  Peachey,  M.  (2006).  Development  of  a  Human  Cognitive  Workload  Assessment  Tool. 
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Assumptions  about  Macrocognitive  Functions 


•  MCF  are  manifested  in  observable  behavior  of  Soldier 
interactions  with  their  environment. 

•  Using  specific  operational  definitions,  they  can  be  recorded 
in  real-time  by  an  SME. 

•  The  co-occurrence  of  MCF  and  workload  extremes  can 
identify  risks  to  MC  performance. 
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Macrocognitive  Functions* 


Code 

Function 

Description 

DM 

Naturalistic 

Decision- 

Making 

The  identification  of  a  feasible  course  of  action  (COA)  from  experience 
accumulated  in  similar  situations;  it  may  involve,  but  does  not  require,  a 
comparison  of  the  strengths  and  weaknesses  of  alternative  COAs. 

S 

Sensemaking 

& 

Situation 

Assessment 

Deliberate,  conscious  process  of  fitting  data  into  a  frame.  The  frame 
may  be  a  story,  script,  map,  or  other  form  of  representation;  the 
intention  is  to  reduce  complexity  and  simplify  the  world  in  relation  to  a 
particular  goal. 

P 

Planning 

Process  of  contemplating  and  devising  actions  for  some  future 
execution  following  a  decision. 

AR 

Adaptation/ 

Replanning 

Once  execution  of  a  plan  begins,  its  progress  is  monitored  in  relation  to 
unfolding  reality.  When  reality  diverges  from  the  plan,  the  plan  may  be 
modified  by  cycling  back  into  the  planning  process,  or  “replanning.” 
Replanning  is  the  process  of  modifying,  adjusting,  and  possibly 
replacing  a  plan. 

PD 

Problem 

Detection 

The  process  by  which  people  first  become  concerned  that  events  may 
be  taking  an  unacceptable  direction;  problem  detection  involves 
consideration  of  actions  that  may  counter  the  perceived  trajectory  of 
events. 

C 

Coordination 

Coordination  is  the  attempt  by  multiple  entities  to  act  in  concert.  Its 
purpose  is  to  achieve  a  common  goal  by  carrying  out  a  shared  script  or 
plan. 

*Developing  the  Soldiers  and  Leaders  of  Objective  Force  and  Beyond ,  Army  Research  Institute  (ARI)  Behavioral  and  Social  Sciences, 
January  2005. 
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Observation  Form 


Macrocognitive  Function 

DM  Naturalistic  Decision-Making  AR  Adaptation/Replanning 

S  Sensemaking/Situation  Assessment  PD  Problem  Detection 

P  Planning  C  Coordination 

Time 

Sender 

Receiver 

Macrocognitive 

Function* 

Work¬ 

load 

(1-5) 

Comments 

DM 

S 

P 

AR 

PD 

C 

The  observation  form  leverages  cognitive  theories  about  workload  and 
mental  activities  to  enable  the  study  of  MC  events. 
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Agenda 


•  Background. 

•  Methods. 

•  Implications  for  MC  Analysis. 
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Workload  Level 


Mock-Up  of  Workload  and  MCF  Analysis 


•  Workload  and  MCF  metrics  were  synchronized  with  time  and  events. 

•  Workload  patterns  were  associated  with  decision-making  and  other  MCF. 

•  Overlays  of  two  echelons’  data  were  examined  for  unit  synchronization. 

•  Synchronizing  time  data  across  observers  was  problematic. 
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Macrocognitive  Function 


Implications  for  MC  Analysis 


•  Creativity. 

-  The  measurement  approach  fosters  spontaneous  behavior  in 
response  to  test  events. 

•  Real-time. 

-  Measures  have  stronger  validity  when  gathered  simultaneously  with 
events. 

•  Objective. 

-  Reduces  risk  of  bias  using  an  anchored  scale  of  visual  protocols. 

-  Both  individual  and  collective  monitoring  are  possible. 

•  Unobtrusive. 

-  Observation  occurs  without  Soldier  interruptions. 

-  Stationary  leaders  are  easier  to  monitor  than  those  on-the-move. 

•  Diagnostic. 

-  Approach  is  more  explanatory  when  workload  is  linked  to 
leader/team  cognitive  functioning  within  MC  systems. 

-  Metrics  are  sensitive  to  changing  context  and  task  demands. 
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